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(54) Title: MULTIPLE FREQUENCY FLUORESCENCE LIFETIME IMAGING 

(57) Abstract 

The application of a high frequency periodic bias 
voltage to the photocathode of an image intensifier device 
results in a repetitive modulation of the gain character- 
istics of the intensifier. By controlling the bias or DC 
component of the bias voltage it is possible to control the 
duration of the 'on* state of the device relative to its *ofT 
state. The amplitude of the higher harmonic content in 
the square waveform can thus be controlled. In this mode 
the image intensifier can be used as a mixing device for 
the simultaneous homodyne detection at each of the har- 
monics. Applied in a fluorescence lifetime imaging mi- 
croscopy (FLIM) application, fluorescence lifetime images 
can be obtained at multiple frequencies in a harmonic set 
calculated from a single series of phase dependent images. 
Both the phase and modulation parameters necessary for 
spatially resolved heterogeneous lifetime calculations are 
extracted by Fourier analysis of the series of phase depen- 
dent images. 
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MULTIPLE FREQUENCY FLUORESCENCE 
LIFETIME IMAGING 



The present invention relates to methods and apparatus for determining true 
5 fluorescence lifetimes in fluorescent materials, and in particular to a 
method and apparatus for imaging spatial variations in such fluorescence 
lifetimes in samples having a composition of varying lifetime species, or 
species in a multiple of lifetime states, distributed throughout the sample. 

10 Temporally resolved spectra offer a wealth of information about the state 
of a fluorescent molecule and its immediate environment. The 
fluorescence lifetime of a molecule is inversely proportional to the sum of 
all the deactivation pathways out of the excited state. Consequently the 
fluorescence lifetime of a fluorophore is sensitive to environmental 

15 conditions such as pH, ionic strength and hydrophobicity, and to excited 
state reactions such as fluorescence resonance energy transfer (FRET), 
molecular quenching and triplet formation. Furthermore, unlike 
fluorescence intensity measurements, the measurement of fluorescence 
lifetime is independent of fluorophore concentration and light path length. 

20 

In recent years, fluorescence lifetime measurements have been applied in 
the microscope for elucidating many aspects of cell physiology, where the 
highly heterogeneous nature of the cell environment makes the 
interpretation of fluorescence intensity especially problematic. 

25 

Fluorescence Lifetime Imaging Microscopy (FLIM) enables the temporal 
attributes of fluorescence emission to be simultaneously measured in every 
pixel of a microscope image and has been described in the prior art applied 
in both the time and frequency domain. 
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In the prior art, frequency domain lifetime measurements in general 
employ a repetitively modulated light source (typically a sinusoidally 
modulated laser source) to excite the fluorophore of interest. The resulting 
fluorescence emitted by the sample is also modulated at the same frequency 
5 but phase shifted and demodulated relative to the excitation source. 

From both the phase shift and demodulation parameters the phase (x^) and 
modulation (x M ) fluorescence lifetimes can be independently calculated. 
Only in fluorescent samples composed of a single homogeneous species are 

10 these quantities equal to each other and to the true fluorescent lifetime of 
the fluorophore. For composite samples the true fluorescence lifetimes can 
only be determined from phase shift and demodulation measurements 
acquired at a number of modulation frequencies. A sample composed of N 
lifetime species requires the phase shift and demodulation to be measured 

15 at a minimum of N frequencies. By fitting these to a pair of dispersion 
relationships the true fluorescence lifetime composition of the sample can 
be estimated. Ideally, the circular frequencies of the modulations should be 
chosen so as to span all the reciprocal lifetimes of the sample. At the high 
frequencies required for nanosecond fluorescence lifetime measurements 

20 (lO's-lOO's of MHz), accurate and precise determination of the repetitive 
fluorescence waveform can be obtained by the application of heterodyne / 
homodyne detection techniques. This type of approach has been extensively 
applied to the lifetime determination of solutions using point detectors i.e. 
lifetime fluorimeters. Homodyne/heterodyne detection using a point 

25 detector can be achieved by mixing the detected fluorescent signal with an 
electronic mixing signal generated either in a lock-in amplifier or by 
mixing with the modulated gain characteristics of the detector directly. 
Typically, such instruments sequentially record phase and modulation data 
at a number of frequencies. 
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In prior an fluorescence lifetime imaging techniques, lifetime 
determination at every pixel of an image is made possible by the use of 
image intensifies as frequency mixing devices. An exemplary prior art 
apparatus and method are described in connection with Figures la and lb, 
5 where figure lb specifically exemplifies the application of the homodyne 
mixing technique. 

A fluorescent sample 1 is irradiated with excitation energy 2 which is 
intensity modulated at a certain frequency by an acousto-optic modulator 3 

10 driven by the amplified voltage output from a frequency synthesizer 4a. 
Fluorescence emission 5 (expressed as F(t) in Figure lb) from the 
irradiated sample 1 is separated from the excitation light via a dichroic 
mirror 12 and an emission filter 13 and is focused onto the photocathode 6 
of an image intensifier having a microchannel plate device 7. 

15 Photoelectrons generated by the light image incident upon the photocathode 
surface 6 are amplified by electron cascade across the microchannel plate 
7, maintaining the spatial resolution of the image. The amplified electron 
image 8 exiting the microchannel plate strikes a phosphor screen 9 to 
generate an amplified light image which can be recorded by imaging onto a 

20 CCD camera 10. 



A second frequency synthesiser 4b, phase-locked to the first and with 
controllable phase 4> G , is used to modulate the gain of the image intensifier 
(expressed as G(t) in Figure lb) either by applying the amplified modulated 
25 voltage signal 11 across the microchannel plate 7 or (as exemplified in 
figure lb) at the photocathode 6. Frequency mixing of the resulting 
modulated gain characteristics with the fluorescence signal is thus 
performed at every pixel of the image output 8 (expressed as D(t) in Figure 
lb). The phosphor screen 9 of the image intensifier behaves as a low pass 

3 
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frequency filter and only the low frequency difference signals are observed 
by the CCD 10 (expressed as D LP (<J> G ) in Figure lb). Where the frequency 
content in the gain modulation G(t) matches that in the fluorescence signal 
F(t), homodyne frequency mixing results in a phase dependent output 

5 image D LP (<J> G ) which contains the phase <|> F and amplitude M F of the 
fluorescence signal. The phase dependent signal D LP ((|> G ) at every pixel of 
the image is sampled over a full phase cycle (0° < <|> G < 360°) by recording 
an image at each phase setting <|> G . From a Fourier analysis of the set of 
phase dependent images D LP (<t> G ) both the phase shift and demodulation 

to associated with the fluorescence signal can be calculated. The minimum 
number of phase samples is chosen to satisfy the Nyquist criterion. 

With prior art FLIM instruments, phase shift and demodulation 
measurements have only been performed at a single modulation frequency 

15 for any one experiment, since sequentially collecting lifetime images at 
every frequency is prohibitively costly, both in terms of data collection 
times and in terms of the total time the microscopic sample is exposed to 
the excitation light. Both these points are especially critical in relation to 
microscopic measurements since the samples under observation are often 

20 dynamic in nature (ie. the movement of fluorescently labelled proteins 
within the spatial environment of a living cell) and far more sensitive to the 
effects of photobleaching than cuvette samples, where large sample 
volumes and diffusion ensure that the fluorophores at the point of 
illumination are continuously replaced. A further consideration is the huge 

25 volumes of data to be processed from multiple sets of phase dependent 
images. 



4 
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It is an object of the present invention to provide a method and apparatus 
for the simultaneous collection of fluorescence lifetime data and images at 
multiple frequencies. 

5 It is a further object of the present invention to provide a method and 
apparatus which enables phase shift and demodulation images at multiple 
harmonic frequencies to be calculated from a single cycle of phase 
dependent images to thereby reduce sample exposure times. The methods 
described herein are equally applicable to the simultaneous collection at 

io multiple frequencies of fluorescence lifetime data from cuvette samples 
using a modulatable point detector ie. by modulating the photocathode 
voltage of a photo-multiplier tube. 

According to one aspect, the present invention provides a method for 
15 making fluorescence lifetime measurements comprising the steps of: 

irradiating a fluorescent material with a beam of excitation energy 
intensity modulated with a plurality of frequencies in a harmonic series 
including a fundamental frequency and at least one harmonic thereof; 

receiving a fluorescence emission from said fluorescent material into 
20 a detector having a controllable gain and an output; 

modulating the output of said detector using said controllable gain, 
with a modulation function having frequency components corresponding to 
at least two of said harmonic series of said irradiating beam, and having a 
controllable phase angle relationship thereto; and 
25 sampling said detector output to determine an amplitude for each of 

a plurality of phase angles. 

According to another aspect, the present invention provides a method of 
imaging spatial variations in fluorescence lifetimes comprising the steps of: 

5 
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irradiating a fluorescent material with a beam of excitation energy 
intensity modulated with a plurality of frequencies in a harmonic series 
including a fundamental frequency and at least one harmonic thereof; 

imaging a fluorescence emission from said fluorescent material onto 
5 a detector having a controllable gain and an output; 

modulating the output of said detector using said controllable gain, 
with a modulation function having frequency components corresponding to 
at least two of said harmonic series of said irradiating beam, and having a 
controllable phase angle relationship thereto; and 
10 generating an image of spatial variation in the modulated detector 

output for each of a plurality of phase angles between said excitation beam 
and said modulation signal. 

According to another aspect, the present invention provides apparatus for 
15 making multi- frequency fluorescence lifetime measurements comprising: 

a source of electromagnetic radiation for irradiating a fluorescent 
material with a beam of excitation energy intensity modulated with a 
plurality of frequencies in a harmonic series including a fundamental 
frequency and at least one harmonic thereof; 
20 means for receiving a fluorescence emission from said fluorescent 

material into a detector having a controllable gain and an output; 

means for modulating the output of said detector using said 
controllable gain, with a modulation function having frequency components 
corresponding to at least two of said harmonic series of said irradiating 
25 beam, and having a controllable phase angle relationship thereto; and 

means for sampling said detector output to determine an amplitude 
for each of a plurality of phase angles. 

6 
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According to another aspect, the present invention provides apparatus for 
imaging spatial variations in fluorescence lifetimes in a sample, 
comprising: 

a source of electromagnetic radiation for irradiating a fluorescent 
5 material with a beam of excitation energy intensity modulated with a 
plurality of frequencies in a harmonic series including a fundamental 
frequency and at least one harmonic thereof; 

means for imaging a fluorescence emission from said fluorescent 
material onto a detector having a controllable gain and an output; 
10 means for modulating the output of said detector using said 

controllable gain, with a modulation function having frequency components 
corresponding to at least two of said harmonic series of said irradiating 
beam, and having a controllable phase angle relationship thereto; and 

means for generating an image of spatial variation in the modulated 
15 detector output for each of a plurality of phase angles between said 
excitation beam and said modulation signal. 



Embodiments of the present invention will now be described by way of 
example, and with reference to the accompanying drawings in which: 

20 Figure la shows essential components of a prior art phase sensitive 

homodyne imaging system and Figure lb illustrates the principles thereof; 

Figure 2 shows a graph (Fig 2b) plotting the photoelectron transfer 
characteristics, in response to constant illumination, of an image intensifier 
device (Fig. 2a); the effective gain response of the image intensifier to a 

25 sinusoidal photocathode voltage (Fig. 2c) is given in Fig. 2d, which can be 
modelled by a square pulse function with adjustable width D (Fig. 2e). 

Figure 3 shows a series of plots (3a to 3e) illustrating the variation 
in the relative modulation depth (the ratio of the amplitude in the gain at 
harmonic n to the average gain, G n /G 0 ) against the fractional pulse width 

7 
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for each of the first five harmonics, calculated from the square pulse wave 
model gain function shown in Fig. 2e; 

Figure 4 shows an exemplary apparatus for carrying out the method 
of the present invention; 
5 Figure 5 shows a series of graphs of experimental data plotting the 

average gain G 0 and relative modulation (G n /G 0 ) for the first four harmonic 
terms in the gain response of an image intensifier measured as a function of 
the relative photocathode voltage bias; 

Figure 6 shows the harmonic content of phase sampled reflected 
10 excitation light for a set of Mow' and 'high' frequency modulations; and 

Figure 7 shows the results of processing mfFLIM data in order to 
separate the fluorescence of two green fluorescent protein mutants co- 
expressed in a live Hela cell. 

15 Figure 1 has already been described in detail, in connection with the prior 
art. Where features of the present invention have corresponding or similar 
function to the features of Figure la, the same reference numerals will be 
used. 

20 The present invention takes particular advantage of the photoelectron 
transfer characteristics of an image intensifier device in order to enable 
control of the higher harmonic content of an image output of the 
intensifier, by square pulse wave modulation of the gain characteristics. 

25 It has been recognised that, because the application of a positive voltage 
applied to the photocathode of an image intensifier acts as a switch to the 
flow of photoelectrons, the application of a zero mean sinusoidal voltage 
results in a square wave modulation of the gain characteristics of the device 
as the sinusoidal voltage alternates in polarity. By controlling the bias or 

8 
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DC component of the sinusoidal voltage, it is possible to control the 
duration of the *on' state of the device relative to its 'off state. 

In this way the amplitude of the higher harmonic content can be controlled 

5 from the width of the square pulse waveform produced. Relative 
modulation depths (the ratio of the amplitude in the gain for the nth 
harmonic component G„ to the average gain G 0 ) of up to 400% are possible 
from this form of modulation. In this mode each of the harmonics in the 
gain of the image intensifier can be used in the simultaneous homodyne 

10 detection of matching harmonic modulations in the fluorescence. The 
means of introducing a matching harmonic set of modulations in the 
fluorescent signal will be discussed below. Thus all the harmonics 
comprising the phase dependent signal at the output of the image intensifier 
device can be sampled for. This requires that the fundamental harmonic 

15 component be sampled at a rate sufficient to satisfy the Nyquist criterion 
for the highest harmonic component present. Ultimately the amount of 
sampling will be limited by the bandwidth of the detector. From a Fourier 
analysis of the single set of phase dependent images the pixel specific phase 
shift and demodulation necessary for spatially resolving heterogeneous 

20 lifetime can be calculated. 

Before a detailed description of the apparatus of the present invention, the 
underlying theory will first be discussed. 

25 Any excitation field (e.g. the excitation energy 2 of Figure la) repetitively 
modulated at frequency /can be represented as a Fourier series: 

£(<) - E Q + £ E n Cos(no>t+Q n ) 0) 



9 
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where co-litf, and is the fundamental circular frequency of the 
modulation, Eq is the time independent average intensity, and E n is the 
intensity amplitude of the n* harmonic frequency component with phase 

5 

After directing the excitation energy 2 at the sample 1, the resulting 
fluorescence emission 5 ( = F(t)) contains the same harmonic content: 

QE is a multiplication factor accounting for photon detection efficiency, 
10 intensity of excitation and fluorophore quantum yield. Here, however, 
each harmonic term of the fluorescence measured relative to the equivalent 
term in the excitation field, has both a phase shift = 0, -©'„ and 
demodulation M n = M F I M E = F n E 0 /E„F Q which vary to an extent dependent 
upon the fluorescence lifetime composition of the sample 1, given by the 
1 5 dispersion relationships : 




where a =a r is the fractional contribution to the steady state fluorescence 

q q q 

from the q* emitting species. The dispersion relationships given by equations 
20 3 and 4 can be fitted at multiple frequencies to resolve lifetimes and 
corresponding amplitudes of samples containing composite fluorescent species 
(Gratton and Limkeman (1983) Biophys. J. 44, 315; Lakowicz and Maliwal 
(1985) Biophys. Chem. 21, 61). 

10 



BNSOOCID: <WO 0008443A1J_» 



WO 00/08443 PCT/G B99/02597 

From these parameters the average phase and modulation dependent 
fluorescence lifetimes can also be evaluated for each of the frequency 
components using the relationships: 

< T?™ >= (5) 



.« £ 



< C >= „ (6) 

nco 



A repetitive high frequency voltage modulation across either the MCP 7 or 
photocathode 6 provided by a frequency synthesiser results in a repetitive 
modulation of the gain characteristics G(t) at every pixel of the detector. 
10 Expanding this into a Fourier series gives: 

G(f 9 k) = G 0 + ]T G m • Cos(ma>t + <p m + mk&<p) (7) 

where G 0 is the average gain amplitude, G m is the gain amplitude for every 
frequency harmonic with associated phase p m and k&<p is the adjustable 
phase setting of the frequency synthesiser, which may be sequentially 
is incremented by &<p. The MCP 7 response is proportional to the incident 
fluorescence intensity multiplied by the gain characteristics of the 
intensifies 



Where both these are given by general equations 2 and 7, the frequency 
20 mixing results in a signal 8 composed of a time invariant response, 
oscillations at the harmonics of the gain and fluorescence modulations, and 
a combination of oscillations at the sum and difference frequencies of the 
harmonics. The slow response time of the phosphor screen 9 at the output 
of the imaging device gives an integrated signal image D(t,k) consisting 
25 only of the low frequency components of the total MCP response: 

n 
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(8) 



where &co = \eo - <y | . 



For the homodyne detection mode the mixing frequencies are chosen to 
give Aw = 0 f resulting in a phase sensitive output image given by: 

I^k)=Q4 i G Q F 0 ^\Y, G nK'Cos(^^nkA<p^ ) (9) 

\ ^ not 7 

Thus from equation (9), a series of images recorded sequentially at discrete 
phases kA<p can be used to map the time dependent fluorescence signal 
given by equation 2. 

Fourier Decomposition and Analysis of mfFLIM phase dependent 
images. 

Linearisation of equation (9) by standard trigonometric identities gives the 
constant (dc), cosine (aj and sine (b^ components of the detected signal, 
is where 



10 



(10a) 



D(k) = dc + ^[a„Cos (nk&<p)- b„Sin (nk&<p)) 



a n =ac n Co^ n ) (10b) 
b n = ac n Sin{</> n ) (10c) 

20 By direct comparison of equations (9) and (10) it is seen that dc = 
QE.G 0 F 0 gives the amplitude of the phase invariant signal and ac 0 = 
QE.GJFJ2 gives the amplitudes of the harmonics. A phase dependent 
image is acquired after every phase step A<p in the gain repeated over a 

12 
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single cycle (0-360°) of the fundamental frequency component of the 
signal. The total number of images is chosen to satisfy the Nyquist 
sampling criterion for the highest significant harmonic component in the 
phase dependent data i.e. k > > 2N. The cosine (aj, sine (bj and dc 

5 terms in the Fourier expansion given by equation 10 may be obtained on a 
pixel by pixel basis from the set of phase dependent images, either by the 
application of a discrete Fourier transform (DFT) or by fitting to a Fourier 
expansion 'model* i.e. a band-limited form of equation 10, using a singular 
value decomposition (SVD) algorithm. For each harmonic the phase shift 

10 A<*„ and demodulation M 0 are calculated from the Fourier amplitudes (dc, 
a„, bn) of the fluorescence signal relative to a zero lifetime sample (ie. a 
scattering or reflective sample) according to: 



15 Where *F £jl and M Zjn are the phase and relative modulation of a reflected 
or scattered excitation light at each harmonic. 

In the above analysis it has been implicitly assumed that both the 
fluorescent signal and gain characteristics of the detector have amplitude in 
20 the high harmonic content. This can be achieved in practice by a number 
of methods. 

Excitation source 





(12) 



13 
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Excitation of a sample under analysis by a beam of excitation energy 
intensity modulated with a plurality of frequencies in a harmonic series can 
be achieved in several ways. 

5 In a first embodiment, a pulsed laser source is used. For instance mode- 
locked lasers typically have pulse repetition rates of approximately 80 
MHz. This is ideal for measuring nanosecond fluorescence lifetimes. The 
pulse width from such a laser is typically less than 50 picoseconds which is 
sufficiently short in comparison to the period (1//) to produce a broad 

10 excitation spectrum which will include significant amplitude in many of the 
higher harmonic terms ie. 160, 240, 320.... MHz. If a lower fundamental 
frequency is required, a mode locked laser can be employed in combination 
with a pulse picker (typically performed using a Pockels Cell). 

15 Alternatively, in a second embodiment, standing wave acousto-optic 
modulators (AOMs) provide a means of modulating a continuous wave 
laser in a sinusoidal manner at high frequencies (e.g. lO's-100's of MHz) 
to produce a much more limited excitation spectrum. Combinations of 
these AOMs can be employed in series to modulate the excitation light with 

20 a combination of frequencies corresponding to the modulation frequencies 
of all the individual AOMs, their difference frequencies and their sum 
frequencies. 

Careful choice of AOM modulation frequency is preferable to ensure that 
25 the frequency content of the excitation corresponds to an harmonic series. 
This approach is used in a preferred embodiment of the present invention 
to be described later in connection with Figure 4. 



14 
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With the technique of this second embodiment, the number of harmonic 
components making up the excitation energy spectrum, E(t) is highly 
constrained. It may then not be necessary to sample the output to satisfy the 
Nyquist criterion for the highest harmonic possible as determined by the 

5 detector bandwidth, merely only to the highest harmonic present in the 
excitation spectrum. Thus, problems associated with aliasing when 
sampling the phase dependent output can be reduced or eliminated. 
Furthermore, the reduction in the sampling requirements has significant 
benefits with respect to lifetime imaging of microscopic samples, where 

10 photobleaching and acquisition times impose significant experimental 
constraints. 

Detector system 

The polarity of any voltage applied to the photocathode of an image 
15 intensifier acts as a switch to the flow of photoelectrons, where a change in 
polarity, from negative to positive, switches off this flow. Thus, in a 
general sense, the application of a periodic zero-crossing control voltage 
results in a square wave type modulation of the gain characteristics of the 
device as the periodic voltage alternates in polarity. Preferably, a 
20 sinusoidal control voltage is used. By controlling the bias, or DC 
component, of the sinusoidal control voltage it is possible to control the 
duration of the 'on' state of the device relative to its 'off state. In this way 
the width of the square pulse waveform can be used to control the 
amplitude of the higher harmonic content. 

25 

It is therefore possible to modulate the output of the detector with a 
modulation function having frequency components corresponding to some 
or all of the harmonic series of the excitation energy irradiating the sample 
under analysis: Preferably, the sinusoidal control voltage has a frequency 

15 
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which corresponds to the lowest (fundamental) frequency of the harmonic 
series of the excitation energy modulation. 

Modulation depths of up to 400% are possible from this form of 
5 modulation. Frequency synthesisers are widely available for providing a 
highly stable, high frequency sinusoidal voltage source (kHz-GHz) on top 
of a controllable bias voltage. 

The determination of photoelectron gain as a function of photocathode bias 
io for an exemplary image intensifier unit (the Hamamatsu C5825) is shown 
in Figure 2. An experimental configuration (corresponding to the image 
intensifier configuration in Figure 1) for the measurement is shown in 
Figure 2a, comprising elements photocathode 6, microchannel plate 7, 
phosphor screen 9 and CCD camera 10. The Hamamatsu C5825 bias 
is voltage is controllable via a ten-turn potentiometer on its power supply 
module from -50V to +25V, with a resolution of approximately ± 0.01V. 

At positive voltages the device is in an 'off state, and a rapid switch to the 
4 on* state occurs for a small negative bias as illustrated in Figure 2b, which 
20 shows the response of the image intensifier device to constant illumination. 
For increasingly negative voltages, after the initial step 20, the gain 
characteristics of the intensifier increase linearly in the portion 21 of the 
transfer characteristic 22. 

25 From the photoelectron gain function thus plotted, the response of the 
image intensifier to the application of a sinusoidal photocathode bias 
voltage 23 as shown in Figure 2c can be shown as that in Figure 2d. This 
response is obtained from the graphical projection of the sinusoidal 
photocathode voltage onto the photoelectron transfer plot. To a good 
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approximation, this represents a square pulse wave type response 24, and 
throughout the present specification, this will be generally referred to as a 
square pulse wave. The width 25 of the square pulse wave modulation can 
be controlled by changing the photocathode bias 26 (Figure 2a). 

5 

A square pulse wave modulation, of arbitrary width D, is given in Figure 
2e. G(t) is periodic in time ie. G(t) = G(t + T) and G(t) = K for 
0<t<D. 

10 Expanding as a Fourier series: 

N 

G(t)= C 0 Co^na* + 9 n ) (10) 

and rewriting using the standard trigonometric identity: 

N 

G(f) = G 0 + Y, Q n Cos(ncot)- b n Sin(nox) (1 1) 

71= 1 

where a„ = G n Cos(ff n ) and b„ = G„Sin(0 H ). (12) 



15 



The Fourier coefficients G 0 , a n and b„ can be evaluated using the following 
set of standard equations: 

G o=jC,m*=j$c.d t as) 

a " = f Irn G(f)Cas(na*ydt = | £ KCo^natydt (14) 
20 b - =| [' T 7 ,G(t)Sin(najt).dt = | £jCSvi(*<i*)d (15) 



Solving the above gives: 

C-f (.6, 

(,7) 
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(18) 



10 



15 



20 



From the sine and cosine components the modulation and phase can be 
determined : 



From the above the relative modulation depth is: 

Mq =$L = 2^ Jl-lCos&nm) (20) 
where r = D/T and is the fractional square pulse width. 

Figure 3 shows the variation in the modulation depth against r for the first 
five harmonic terms of the gain, where G 0 is the fundamental, and G ; ...G 5 
are the first five harmonics. 

Relative modulation depths up to 400% are possible for sufficiendy narrow 
pulse widths. This is four times greater than the maximal modulation 
depth possible with pure sine wave modulation. The higher modulation 
comes at a cost to detection sensitivity. 

With existing technology, detectors can operate satisfactorily with 
modulated outputs as high as 360 MHz. 

With reference now to Figure 4, a presently preferred embodiment of a 
multiple frequency fluorescence lifetime imaging apparatus 40 will now be 
described. 




and 
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Instrument Configuration 

The continuous wave output 42 of an Argon/Krypton laser 41 is passed 
through a pair of AOMs 43, 44 (either a 40 MHz or 160 MHz in 
combination with an 80 MHz standing wave AOM 44). The output light 
5 source excitation beam 45 is thereby sinusoidally modulated at high 
frequencies at both the fundamental AOM frequencies, and the sum and 
difference frequencies, to provide a harmonic set of modulation frequencies 
from 20 MHz to 340 MHz, which are used to excite sample 1 within 
microscope 71. 

10 

The AOMs 43, 44 are respectively driven by the amplified voltages 50, 51 
from two (slave) frequency synthesisers 52, 53 which are phase locked to a 
third (master) frequency synthesiser 60 using a first output 60a. The third 
frequency synthesiser 60 also has a second, phase-adjustable, output 60b. 
is This is used to derive, via amplifier 61, a gain control signal 62 used to 
modulate the photocathode voltage of an image intensifier 70, at the 
fundamental harmonic frequency of the excitation beam 45. 

(It will be understood that the photocathode voltage could alternatively be 
20 modulated at one of the harmonic frequencies of the excitation beam, but 
that this would reduce the number of higher harmonics in the gain response 
G(t) for which lifetimes can be calculated, and would thus be less 
efficient.) 

25 The relative phase difference between the excitation beam 45 and the gain 
control signal 62 can be controlled by the phase-adjustable output 60b, 
enabling phase dependent image collection. This is preferably carried out 
under the control of a computer system 81 by control bus 82. 
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Iris diaphragms 46, 47 placed approximately 1.5 metres from each of the 
AOMs 43, 44 select the zero from the higher order diffracted beams (6.4 
mrad beam separation) and a variable neutral density wheel 48 of 0-5 OD 
provides for control of the overall signal intensity. 

5 

To eliminate the effect of laser speckle at the sample 1, the high spatial and 
temporal coherence properties of the laser beam are removed by passing 
the light through a rotating ground glass disc 49. 

10 The scattered radiation from the rotating ground glass disc 49 is collected 
and collimated with a high numerical aperture lens 58 before being directed 
into the epi-illumination port of an inverted microscope 7 1 which results in 
Kohler illumination at the sample 1 . 

15 Steady state fluorescence images are captured using a scientific grade CCD 
camera 10. For recording lifetimes, the sample fluorescence is first 
imaged onto the photocathode 9 of a C5825 image intensifier (Figure 2a). 
A telescopic lens 72 optically couples the phase dependent image at the 
phosphor screen 9 output to the CCD camera 10 and images are 

20 downloaded over an AIA bus 80 to an interface card in the computer 
system 81. 

Preferably, the computer system 81 also provides positional control and 
feedback of a motorised stage 83 on which the sample is mounted, by way 
25 of a control unit 84 and interface bus 85. 

A series of phase dependent images are taken over the full cycle (0-360°) 
of the fundamental harmonic, where the number of images satisfies 

20 
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present in the homodyne 



The resulting sampled output is then passed to the computer system 81 
5 where a Fourier analysis can be performed. 

Figure 5 shows exemplary measurements of relative harmonic content 
present in the gain characteristics of the C5825 image intensifier as 
function of photocathode relative bias voltage. The relative bias voltage is 

10 defined as the photocathode offset voltage divided by the amplitude of the 
applied sinusoidal voltage. These results were obtained by measuring a 
cycle of phase dependent images from a reflecting sample 1 illuminated 
with pure sinusoidally modulated laser light which is an integer multiple of 
the driving frequency of the image intensifier. The relative amplitude at 

is each harmonic is obtained by Fourier analysis. A photocathode bias signal 
comprising a ±17.8 V sinusoid plus a variable bias is applied to the 
photocathode 6. The average gain in Figure 5a has been normalised to its 
maximum value and in Figures 5 (b-e), the relative modulation depths in 
the gain are scaled by half the relative modulation depth of the reflected 

20 excitation. The data for each figure was acquired with the fundamental 
frequency of the reflected light and photocathode voltage respectively set to 
(a, b) 80.236 and 80.236 MHz, (c) 80.236 MHz and 40.118 MHz, 
(d)80.238 MHz and 26.746 MHz and (e)80.236 MHz and 20.059 MHz. 

25 Typical examples of phase sampled excitation light from a reflecting 
sample and its relative harmonic are shown in Figure 6. Because the 
sample (a piece of aluminium foil) has no lifetime associated with it these 
figures essentially show the instrumental response of the detector to the 
excitation modulations. The data shown in Figure 6a, designated as the 
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"low" frequencies set,was obtained with the 40MHz and 80MHz AOMs 
43, 44 set to modulate the excitation light at frequencies of 42.154 MHz 
and 63.231 MHz respectively; where frequency mixing gave rise to 
additional modulations with amplitudes greater than 0.1 at 21.077MHz, 

5 105.385MHz and 126.462MHz. The data shown in Figure 6b, designated 
as a "high" frequency set, was obtained with the 80MHz and 160MHz 
AOMs 43, 44 set to modulate at a frequencies of 80.244 MHz and 160.488 
MHz respectively, giving additional modulations in intensity with 
amplitudes greater than 0.1 at 240.732 MHz and 320.976 MHz. The 

io relative bias on the photocathode of the image intensifier was set at about 
0.9. The upper figures of Figure 6a and b plot the average intensity from 
the phase-sampled images of the reflective sample. The corresponding 
harmonic content in the data obtained, from a Fourier analysis, is shown in 
the lower figures respectively. 

15 

The mfFLIM configuration and fitting routines were tested on an equi- 
Molar (1 ^iM) solution of rhodamine 6G and rhodamine B in distilled 
water. The excitation light was modulated by the 40MHz and 80Mhz 
AOMs tuned 42.154 MHz and 63.231 MHz (including the sum and 

20 difference mixing frequencies). Homodyne detection of all the excitation 
frequencies was achieved by modulating the photocathode of the MCP at a 
fundamental frequency of 21.077 with a relative voltage bias of about 0.9, 
and 32 phase sampled images were collected in each case (i.e. 11.25° phase 
steps between images). Reference images were obtained by phase 

25 sampling reflected light from aluminium foil illuminated with the 
modulated laser light. All phase dependent images were corrected for dark 
current and stTay light by subtracting an image acquired in the absence of 
excitation field illumination. No significant photobleaching was observed 
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in the fluorescence signal, due in main to the large difftisional volume of 
the sample. 



Table 1 lists the results of fitting the phase shift and demodulation images 
5 generated from a Fourier analysis of the set of phase dependent images to 
the dispersion relationships (equations 3 and 4), based on a bi-exponential 
lifetime model. The lifetime values of 1.29 ns and 3.74 ns correspond well 
with listed literature values of 1.5ns and 4ns for rhodamine B and 6G 
respectively (Lakowicz & Berndt (1991). Rev. Sci. Instrum. 62, 1727). A 
io relative intensity contribution of 73% was recovered for the fluorescence 
from rhodamine 6G (c^) which was in excellent agreement with a 71% 
value calculated from independent intensity measurements of rhodamine 6G 
and rhodamine B alone using the same filter set as for the mfFLIM 
experiment. 

15 

Table 1 : Fluorescence lifetime fit parameters of rhodamine B/6G mixture. 





avg 1 


sdev 2 


<err> 3 


a, 


0.73 


0.15 


0.15 


x, (ns) 


1.29 


0.52 


0.53 


t 2 (ns) 


3.74 


0.41 


0.39 



It will be noted that the number of frequencies in the harmonic series 
20 present in the excitation and gain modulation spectrum should be at least as 
many as the number of different lifetime species or states expected in the 
sample 1 in order to be able to resolve each lifetime species or state by 
fining the data to the dispersion relationships. 
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In preferred modes of use, the excitation spectra of the detector systems of 
the present invention are calibrated by using a scattering surface sample 1 
having no fluorescence lifetime characteristics. This enables determination 
of the optical and electrical path lengths of the detection system and the 
5 phase and modulation shifts caused thereby. This information is stored by 
the computer system 81 and used to compensate the output measurements 
taken. 

Finally Figure 7 shows a practical application of the mfFLIM instrument. 

10 Here it is shown that mfFLIM can be used to disentangle the cellular 
distributions of two co-expressed green fluorescent protein mutants with 
differing lifetimes. In Hela cells a construct of the Golgi localisation signal 
N-acetylylglucosaminyltransferase I fused to the green fluorescent protein 
mutant GFP5 (ZernickaGoetz et al. (1997), Development 124: 1133). 

15 (referred to as NA-GFP5) was co-expressed with the novel green 
fluorescent protein mutant YFP5 (which will distribute in the cytosol and 
nucleus). YFP5 is described in UK Patent co-pending application entitled 
"Fluorescent Protein" which has the same filing date as this application. 
Because of a high degree of spectral overlap it is difficult to efficiently 

20 isolate the fluorescence of these two GFP mutants by conventional 
filtering. In mfFLIM measurements a long pass dichroic and broadband 
emission filter can be used in order to simultaneously collect the 
fluorescence from both the GFP mutants. Figure 7a shows the dc 
component of the set of phase dependent images taken of one of the 

25 expressing Hela cells. Figures 7b and 7c show the isolated fluorescence 
from the GFP5 and YFP5 respectively, calculated by fitting the phase shift 
and demodulation images to the dispersion relationships based on a bi- 
exponential decay model. 
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Improving mfFLJM data analysis 

Multiple parallel measurements of the time-resolved fluorescence kinetics 
of the probes in different states are obtained in mfFLIM ie. each pixel in 
the image can be conceived of as an individual experiment. Global analysis 

5 of a multiple of such experiments has a clear advantage over individual 
analysis of the data at a single point (Beechem (1992) Methods Enzymol. 
210, 37). Inter-relationships between decay parameters at each pixel can be 
encoded in a global fit of the image in order to significantly reduce the 
fitting errors. For example, for a mixture of two fluorophores with 

10 homogeneous decays or a single fluorophore in two states, the lifetimes are 
pixel invariant and could be linked in the double exponential decay fitting 
model over the whole image where the amplitudes are left uncoupled- This 
results in a reduction of 2N 2 -2 fitting parameters for an NxN image 
Furthermore by using a global fitting approach with mfFLIM data it should 

15 be possible to significantly reduce the number of frequency measurements 
necessary to achieve estimates of fluorescent lifetime parameters with a 
specified signal to noise ratio. This is critical with microscopic samples 
where the data acquisition time and total exposure have to be carefully 
controlled. The measurements of the populations of GFP fusion proteins 

20 or fluorescence resonance energy transfer (FRET) through donor and 
acceptor tagged proteins in cells would be examples of systems where the 
global fitting approach might be expected to bring significant 
improvements in quantifying the populations or states of molecules. 

25 
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1. A method for making fluorescence lifetime measurements 
comprising the steps of: 

5 irradiating a fluorescent material with a beam of excitation energy 

intensity modulated with a plurality of frequencies in a harmonic series 
including a fundamental frequency and at least one harmonic thereof; 

receiving a fluorescence emission from said fluorescent material into 
a detector having a controllable gain and an output; 
io modulating the output of said detector using said controllable gain, 

with a modulation function having frequency components corresponding to 
at least two of said harmonic series of said irradiating beam, and having a 
controllable phase angle relationship thereto; 

sampling said detector output to determine an amplitude for each of 
15 a plurality of phase angles. 

2. A method according to claim 1 further including the step of: 
selecting the number of frequency components in said harmonic 

series to be at least as many as the number of different lifetime components 
20 to be measured in the fluorescent material. 

3. A method according to claim 1 or claim 2 in which the step of 
irradiating the fluorescent material includes excitation with a pulsed laser 
source to generate a broad modulation spectrum inclusive of components at 

25 each frequency of said harmonic series. 

4. A method according to claim 1 or claim 2 in which the step of 
irradiating the fluorescent material includes the step of modulating a 
continuous wave laser source using a plurality of acousto-optic modulators 
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selected to generate said harmonic series with their individual, sura and 
difference frequencies. 



5. A method according to any preceding claim wherein the detector 
5 includes an image intensifier device and in which the modulating step 
comprises the step of applying a periodic bias voltage to the photocathode 
of the image intensifier so as to result in square wave modulation of the 
gain characteristics of the image intensifier device. 

io 6. A method according to claim 5 in which the periodic bias voltage 
comprises a sinusoidal voltage having a peak-to-peak voltage and a DC 
component sufficient to cause negative-going excursions for only a small 
portion of its negative half-cycle. 

15 7. A method according to any preceding claim in which the sampling 
step includes the step of sampling the detector output over a range of phase 
angles relative to the lowest frequency component of the modulation signal, 
the number of sampled phase angles being sufficient to resolve the highest 
frequency component of the harmonic series in the detector output. 

20 

8. A method according to claim 7 in which the number of components 
in the excitation energy is substantially restricted to only said harmonic 
series so as to reduce sampling requirements. 

25 9. A method according to claim 7 in which the number of sampled 
phase angles is sufficient to resolve the highest frequency component of the 
detector. 
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10. A method according to claim 1 further including the step of using 
said sampled output to determine the phase and modulation lifetimes of said 
fluorescent material at each of said modulation signal frequencies, by: 

carrying out a Fourier transform on said sampled output of the 
5 detector to determine the amplitude of the DC and AC components of the 
harmonic series; 

for each frequency of the harmonic series, determining the phase 
shift and demodulation; and 

for each frequency of the harmonic series, evaluating the phase and 
io modulation lifetimes. 



11. A method according to claim 1 further including the step of 
determining the lifetime composition of the sample by the steps of: 

carrying out a Fourier transform on said sampled output of the 
15 detector to determine the amplitude of the DC and AC components of the 
harmonic series; 

for each frequency of the harmonic series, determining the phase 
shift and demodulation; and 

fitting the phase shifts and demodulations to the dispersion 
20 relationships of equations (3) and (4). 

12. A method according to claim 10 further including: 
simultaneously carrying out the steps of claim 10 for each of a 

plurality of spatial locations in the fluorescent material; and 
25 simultaneously determining the spatial variation in phase and 

modulation lifetimes of said fluorescent material. 



13. A method according to claim 12 in which the fluorescent material 
comprises a mixture of at least two components each having a 
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predetermined interrelationship between decay parameters, in which said 
step of simultaneously determining the spatial variation in phase and 
modulation lifetimes includes using a global fitting algorithm incorporating 
said predetermined interrelationship to reduce fitting errors. 

5 

14. A method according to claim 13 in which said predetermined 
interrelationship includes said fluorescent material components each having 
a lifetime which is invariant across said plurality of spatial locations. 

10 15. A method according to claim 12 in which the detector includes an 
image intensifier including a microchannel plate device, the method further 
including the steps of: 

imaging the fluorescent material onto the photocathode of an image 
intensifier; 

15 simultaneously amplifying the photocathode output for each of a 

plurality of pixels of the image using the microchannel plate device; 

determining the amplitude of the output for each of the plurality of 
phase angles for each pixel of the image; and 

using said sampled output to determine the phase and modulation 
20 lifetimes of said fluorescent material at each of said modulation signal 
frequencies for each pixel of the image. 

16. A method of imaging spatial variations in fluorescence lifetimes in a 
sample comprising the steps of: 
25 irradiating a fluorescent material with a beam of excitation energy 

intensity modulated with a plurality of frequencies in a harmonic series 
including a fundamental frequency and at least one harmonic thereof; 

imaging a fluorescence emission from said fluorescent material onto 
a detector having a controllable gain and an output; 
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modulating the output of said detector using said controllable gain, 
with a modulation function having frequency components corresponding to 
at least two of said harmonic series of said irradiating beam, and having a 
controllable phase angle relationship thereto; and 
5 generating an image of spatial variation in the modulated detector 

output for each of a plurality of phase angles between said excitation beam 
and said modulation signal. 

17, Apparatus for making multi-frequency fluorescence lifetime 
io measurements comprising: 

a source of electromagnetic radiation for irradiating a fluorescent 
material with a beam of excitation energy intensity modulated with a 
plurality of frequencies in a harmonic series including a fundamental 
frequency and at least one harmonic thereof; 
15 means for receiving a fluorescence emission from said fluorescent 

material into a detector having a controllable gain and an output; 

means for modulating the output of said detector using said 
controllable gain, with a modulation function having frequency components 
corresponding to at least two of said harmonic series of said irradiating 
20 beam, and having a controllable phase angle relationship thereto; 

means for sampling said detector output to determine an amplitude 
for each of a plurality of phase angles. 

18. Apparatus for imaging spatial variations in fluorescence lifetimes in 
25 a sample, comprising: 

a source of electromagnetic radiation for irradiating a fluorescent 
material with a beam of excitation energy intensity modulated with a 
plurality of frequencies in a harmonic series including a fundamental 
frequency and at least one harmonic thereof; 
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means for imaging a fluorescence emission from said fluorescent 
material onto a detector having a controllable gain and an output; 

means for modulating the output of said detector using said 
controllable gain, with a modulation function having frequency components 
5 corresponding to at least two of said harmonic series of said irradiating 
beam T and having a controllable phase angle relationship thereto; and 

means for generating an image of spatial variation in the modulated 
detector output for each of a plurality of phase angles between said 
excitation beam and said modulation signal. 
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